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Cost and Quality Planning for Large NASA Programs 


The Software Cost and Quality Engineering methodology developed over the last two decades at IBM 
Federal Sector Division (FSD) in Houston is used to plan the NASA Space Station Data Management 
System (DMS). An ongoing project to capture this methodology, which is built on a foundation . ■ 
experiences and "lessons learned,” has resulted in the development of a PC-based tool that integrates cost 
and quality forecasting methodologies and data in a consistent manner. This tool, Software Cost and 
Quality Engineering Starter Set (SCQESS), is being employed to assist in the DMS costing exercises. A: the 
same time, DMS p lannin g serves as a forcing function and provides a platform for the continuing, it era t i ve 
development, calibration, and validation and verification of SCQESS. The data that forms the cost and 
quality engineering database is derived from more than 17 years of development of NASA Space Shuttle 
software, ranging from low criticality, low complexity support tools to highly complex and highly critical 
onboard software. 


INTRODUCTION 

Software cost and quality engineering is the systematic approach to the estimation, measurement, and 
control of software costs and quality on a project. This discipline provides the vital Imk between the 
concepts of economic analysis and the methodology of software engineering. The tasks involved in software 
cost and quality engineering are complex, and individuals with the knowledge and skill required are scares. 
The accuracy and consistency of the results are often questionable. There is a definite need for tools to 
enable analysis by managers and planners who are not experts and to improve the results. 

There arc many instances in planning software development activities when a quick and easy to use cost and 
quality estimating tool wouid be of value. These include management consulting, proposal generation, and 
analysis of existing programs for problem correction or. assurance. There is little time to learn a complex 
tool or to digest introductory user information. SCQESS requires no set-up, has a selectable demonstration, 
direct access to the tool functions, and contains examples of varied applications. 

SCQESS resides on a tool sharing disk which is available to all company sites. The user has the option of 
viewing a demonstration program to illustrate the cost and quality estimation process. SCQESS includes a 
Lotus-based cost and quality estimation spreadsheet. The spreadsheet inrlitrics cost and quality models 
which are based on historical data and various criticality levels. If different models are required, the existing 
mode’s can be easily modified. An example Lotus spreadsheet illustrates a completed estimation. This 
spreadsheet can be modified or a blank file is supplied if the work is entirely new. The tool can handle 
estimates involving many languages including Ada and can also be used for estimating reused elements. 

Once the basic cost and quality estimates are completed, the user executes a Rayleigh Curve program to 
phase the estimates over time. A Rayleigh curve is a plot of a mathematical function which describes Efe 
cycle phenomena. A Rayleigh curve indicates whether the slope of the staffing curve is too steep or whether 
the error density of the project is too great at certain points in the process. 

The cost and quality estimate can then be quickly modified to determine variance of results based on 
changes in assumptions. The cost and quality esti m at es support the strength of the plan being g en e ra ted. 
The estimates can be used to analyze ride and mitigation plans. Examples of actual project cost and quality 
reports are available to the user. These examples include a project involving Ada. reuse and commercial 
elements; a project involving translation of code from one machine architecture to another: and a transaction 
oriented commercial system. The spreadsheets and the Rayleigh curves are included in the report. 
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Software Life Cycle Costing and Quality Estimation Methodology 

Past experience in managing large software projects such as Space Shuttle Onboard Software illustrates thx 
accurate cost and quality estimates based on reliable historical data are essential to software planning. PSD 
Houston has co llect ed extensive data from NASA and other software development projects over the tast IT 
years. This data includes source lines of code, productivity, error rates, computer usage, esc. for more ihar 
250 projects. Historical data supports initial estimation of project size and estimation of effort from thaa 
size. The standard Rayleigh curve model converts estimated labor to a schedule and staffing profile - the 
elements of a cost plan. It also projects error estimates across the schedule to generate a quality plan. 

The widely used Rayleigh curve models the typical build-up of staff and errors during the requirements and 
design phases, the peak for implementation, and the tail-off during the testing phase. The staffing schedule 
for an ideal project approximates a Rayleigh curve. During sustaining engineering, a minimum lewel a 
critical skills is required for effective maintenance. This steady-state staffing level forms the support lime. ~ 
includes critical stalls for requirements, design, implementation, testing, and management. The support lire 
is a function of system size and productivity as well as unique skill requir e me n ts specific to the -.orirwaie 
being maintained. 

The areas below the support line and above the maintenance tail of the Rayleigh curve is available fear rarer 
development work. Sustaining engineering operational increment also corres p onds to a Rayfczh curve. 
Each sustaining engineering effort can be modeled as the sum of a sequence of such curves. The sizing an* 
scheduling of new development activities should be planned to provide a stable level of effort. Soc man : 
maintenance which handles Discrepancy Reports can continue at a lower support level. 

Historical project data supports software size, labor, and quality estimation. The Lceus-based Matrix 
Method function distributes the effort and errors over organizational elements. The Rayleigh Curve function 
generates a staffing and error discovery profile over time. Special models are available in other packages :a 
adjust estimates involving expert systems, reusable software, reconfiguration, qualiry tracking, acc 
maintenance. 


Planning the Space Station DMS Utilising SCQESS 

SCQESS has been used to assist in the costing of the Space Station Data Management System (DMS). x 
complex software system involving a distributed environment with multiple languages and applications. The 
DMS for Space Station is also affected by the requirements for long lifetime, per manent operations, remote 
integration, and phased technology insertion of productivity tools, applications, expert systems, etc. Major 
cost and quality drivers include the large size and diversity of the software, complexity, development stepper: 
environment, off-the-shelf and reusable software, and criticality, which varies from one module to another. 
An example of the type of results — at the end of the interme di a t e step of development cost and oualirv 
estimation — obtained with SCQESS for the DMS planning is included in the prcseniation. 


Summary/Condusions 

The software cost and quality engineering methodology employed at IBM FSD Houston has been captured 
and integrated into a prototype tool, SCQESS. This PC-based tool integrates cost and quality planning 
methodology and data. 

SCQESS has been employed to assist in the cost and quality planning of the Space Station DMS (Daa 
Management System). It is providing a standardized approach for the DMS planning, w hich involves s e v er ri 
individuals. It has made the process more efficient and has allowed a consistent approach to p fennin g The 
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automation and captured methodology has established the foundation and mechanism enabling the 
continuing calibration and improvement in accuracy and consistency for Space Station DMS costing. 
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Keys to Customer Satisfaction 


• Compliant Product 

• Within Budget 

• On Time 

• Appropriate Quality Level 
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MODEL RECONCILIATION 


o ALT (EARLY MODEL) 

PROJECTED = 10429 MM 

DEVELOPMENT PART OP ALT = 9403 MM ACTUALS 
ERROR IS 1026 MM OR 11% HIGH 

o STS-1 (MIDDLE MODEL) 

PROJECTED = 8905 MM 

DEVELOPMENT PART OF STS-1 = 9521 MM ACTUALS 
ERROR IS 616 MM OR 6% LOW 

o STS-2 THRU STS-5 (MIDDLE MODEL) 

PROJECTED •* 5864 MM 

DEVELOPMENT PART OP STS-2 THRU STS-5 = 5994 MM ACTUALS 
ERROR IS 130 MM OR 2% LOW 

o TOTAL 

TOTAL PROJECTED » 10429 + 8905 + 5864 - 25198 MM 
TOTAL ACTUALS - 9403 + 9521 + 5994 - 24918 MM 
p - ERROR IS 280 MM OR 1% HIGH 

r 

O flTS-1 'i'll HU 8T0-5 (MIDDLE MODEL) 

TOTAL STS-1 THRU STS-5 PROJECTED ~ 8905 + 5864 - 14769 MM 
TOTAL ACTUALS » 9521 + 5994 - 15515 MM 

KHH0H 18 146 MM OH 51 



MODEL RECONCILIATION 


SDL 

(900,000/230) 1.4 = 5478 MM 
SDL ACTUALS « 5730 MM 
ERROR IS 252 MM OR 4% LOW 

SPF 

((250,000/230) + (315,000/250) + (385,000/475)) 1.4 = 4421 MM 

SPF ACTUALS *= 4033 

ERROR IS 388 MM OR 10% HIGH 

TOTAL 

TOTAL PROJECTED = 5478 + 44 21 *= 9899 MM 
TOTAL ACTUALS «= 5730 + 4033 = 9763 MM 
ERROR IS 136 MM OR 1% HIGH 
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ERROR DISCOVERY PROFILE FOR PROJECT 04-13-1990 


SOFTWARE 

DEVELOPMENT ACTIVITY 

ERRORS 

USER PROVIDED 

PER KSLOC 

PROGRAM EST* 

HIGH LEVEL DESIGN INSPECTION 

0.11 

1 . 0062 

LOW LEVEL DESIGN INSPECTION 

4.20 

3. 1&67 

CODE INSPECTION 

6.ao 

6.9371 

UNIT TEST 

— 

:,721 

integration test 

— 

2.011 

SYSTEM TEST 

— 

0.6S4 

LATENT ERROR 

— 

0.598 


THE PATTERN PROJECTED IS PATTERN NUMBER: 14 

THE ESTIMATED TOTAL LIFETIME ERROR CONTENT IS: 18.29 

PRESS: ENTER FOR MENU SCREEN ?PrtSc S. Shi -ft TO PRINT SCREEN: Y TO GRAPH DAT- 
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PROJECTION COMPARISON 


STEER ACTUAL 

• TOTAL RELEASE 

- TOTAL INSERTED ERROR RATE 18.3 13,3 

- PRODUCT ERROR RATE .6 1.0 


• HOST 

- TOTAL INSERTED ERROR RATE 13.0 8. 

- PRODUCT ERROR RATE .4 


• W/S 

- TOTAL INSERTED ERROR RATE 35.1 28.3 

- PRODUCT ERROR RATE 1.1 3.2 
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PROJECTION COMPARISON 
(INCLUDING PROCESS DATA) 


TOTAL RELEASE 

STEER 

ACTUAL 

- TOTAL INSERTED ERROR RATE 

13.54 

13.28 

- PRODUCT ERROR RATE 

1.25 

1.0 

HOST 

- TOTAL INSERTED ERROR RATE 

8.47 

8.52 

- PRODUCT ERROR RATE 

.28 

.33 

W/S 

- TOTAL INSERTED ERROR RATE 

28.28 

28.82 

- PRODUCT ERROR RATE 

2.62 

3.16 
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Software Ufa Cyde Costing (LCC) Methodology 
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((SLOC/PROD) 1.6) 2.6 


23 ERRORS/KSLOC EARLY 
6.7 ERRORS/KSLOC PROCESS 
.1 ERRORS/KSLOC PRODUCT 


CRITICALITY 


OEV. = (SLOC/PROD) 1.4 


21 ERRORS/KSLOC EARLY 
B ERRORS/KSLOC PROCESS 
1 ERROR/KSLOC PRODUCT 


PRODUCT ERROR RATE 
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REQUIREMENTS 
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SOFTWARE COSTING METHODOLOGY 
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r 
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5000 
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30.8 
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2000 
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33.9 
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r 
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